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• Abstract 

. Written expression samples from 50 students in grades 3-6 were 
scored in terms of correct word sequences to investigate (a) the 
^consistency among scorers using the procedure, (b) the typical 
performance levels of students in grades 3-6 on this measure, and (c) 
the validity of this measure relative to criterion measures of written 
expression. Analyses revealed that the average inter-scorer agi;'eemen:t / 
was 9Q.3<. Average scores for students in grades 3-6 ranged from 277J "^^^ 
?t grade 3 to 58.8 at grade 6, with an increase of about 10 for each 
successive grade level.. Correlations between correct word sequences 
and several criterion measures, including a holistic rating, were very 
high. Implications of the results for use of the correct word 
sequences measure within a formative evaluation system are discussed. 
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Correct Word Sequences: A Valid Indicator of 
Proficiency in Written Expression 

In order for teachers to del iver ^appropriate instructional 
strategies in written expression, they must have available to them an 
efficient method of accurately measuring their students' progress. 
Lindsley (1971) states that what teachers .need most is a way ^of 
comparing and evaluating the daily effects of their teaching 
procedures. There are a variety of methods for evaluating .written 
expression. Many of these evaluation procedures are subjective in 
nature, difficult to score, or require expertise and special training. 

For many aspects of written expression, there are no satisfactory 
standard.ized tests that can be used to evaluate a child's performance 
(Hammill &.Bartel, 1978). Further, standardized tests have been 
criticized as being biased and inappropriate for individual assessment 
(Ysseldyke, 1979). A fairly new achievement test, Hammill and 
Larsen's (1978) Test of Written Language, is promising, although it 
requires special training, and is inappropriate for use in formative 
evaluation (Poplin, Gray, Larsen, Banikowski, & Mehring, 1980). 

Formative evaluation, according to Scriven (1967), is "the 
outcome evSluation of an intermediate stage in the development of the 
teaching instrument" (p» 51). Through the use of feedback, formative 
evaluation, unlike summative evaluation, leads to the improvement of 
the instruction during the instructional process with the student 
(Mirkin & Deno, 1979) • This is extremely important since students can 
only take advantage of improved instructional techniques during their 
instruction. 



General support for formative evaluation methods has already been 
obtained (Boucher, 1982; Cooper & Johnson, 1979; Crutcher & 
Hofmeister, 1975; Lovitt, Schaff, & Sayre, 1970; 'starl in, 1970; White 
It Haring, 1976). When formative evaluation is to be used in improving 
a program, however, specific measures of student performance must be 
used. Since how to assess written expression was unclear, a series of 
research studies was conducted (Deno, Mirkin, Lowry, & Kuehnle, 1980; 
Deno, Mirkin, & Marston, 1980; Marston l Deno, 1981; Marston, Lowry, 
Deno, & Mirkin, 1981) to determine both what teachers should measure 
and how tney should measure to reliably, validly, and efficiently 
monitor increases in writing proficiency. The results of the research 
revealed that production of words and letter sequences in response to 
story startfers validly indexed proficiency in written expression 
(Deno, Mirkin, & Marston, 1980). 

Despite the research data regarding the technical adequacy of 
word production as a measure of written expression, two issues were 
not addressed adequately by Deno, Mirkin, and Marston (1980). First, 
in their research no effort was made to relate written language 
production to holistic impressions of students' writing samples. 
Since holistic ratings are viewed with favor by some researchers 
interested in written expression (Cooper, 1977; Lloyd-Jones, 1977), 
the relationoship between word production and holistic impressions 
needs to be e3(amined. 

A second issue is whether students will begin to rather 
arbitrarily, or randomly, generate additional words when they discover 
that teachers are scoring their written samples in terms of total 



words written. Thus, while initial sample scores for production of 
written language might validly index skill in written expression, 
repeated measurements might result in spurious increases in number of 
words written that are unrelated to real improvement in. written 
language proficiency. The present study .was designed, in P.art, to 
address this issue. 

The purpose of the study presented here was to determine whether 
counting correct word sequences is a valid measure of written 
expression for use in a formative evaluation System for elementarv 
students. Several major aspects of writing,, including capitalization, 
punctuation, grammar, spelling, and content or meaning (Hammill & 
Bartel, 1978; Marston & Deno, 1981; Slotnick, 1973), were incorporated 
into the counting correct words sequences measure,. The specific 
questions addressed were: 

(1) Can elementary students* samples of written expression 
be scored consistently in terms of the number of 
correct word' sequences? ^ 

(2) What is the topical performance o'f .elementary students, 
grades 3-6, in writing correct word sequences? 

(3) What is the relationship between writing correct word 
sequences and other criterion measures of written 
expressio,n, including holistic impressions? 

For this study, "written expression" was defined, as the use of 

conventional English language to convey thoughts and experiences 

through a standard graphic symbol system utilizing acceptable 

capitalizatiqn, punctuation, grammar, and spelling competencies. A 

"correct words^quence" was defined as two adjacent, correctly spelled 

words that .are acceptable within the context of the phrase to a native 

speaker of the English language'. 



' Method . 

Subjects 

The subjects for this study were 50 students whose files were 
randomly selected from 135 elementary children who had participated in 
a study conducted by Oeno, Mirkin, and Marston (1980). The students 
were in grades three through six (ages 7-11) and had been selected 
randomly from seven elementary schools in the Twin Cities metropolitan 
area. 

Oeno, Mirkin, and Marston (1980) administered the reading, 
mathematics, and written language achievement subtests from the 
Woodcock-Johnson Psycho-Educational Battery (Woodcock & Johnson, 1978) 
to a subsample of 31 of their subjects. Reading. Achievement, Math 
Achievement, and Written Language Achievement grade scores were 
computed for the 31 students tested. The average age of these 
students was 120.1 months and their average grade level was 4.7. 
Their average grade level scores were 4.6, 4.7, and 4.6 on the 
reading, math, and written language acheivement measures, 
respectively. 

Table 1 specifies the number of males and females., per grade 
level, whose written expression samples were analyzed in the present 
study, as well as their average raw score totals 'on the Test of 
Written Language (TOWL) (Hammill & Larsen, 1978). 



Insert Table 1 about here 



Seven criterion variables were used to establish the concurrent 
validity of Correct Word Sequences as a measure of written expression. 
In order for a measure to be acceptable for use as a validation 
criterion, that measure also rnust be reliable and valid (Mehrens & 
Lehmann, 1978; Salvia I Ysseldyke, 1978). According to the American 
Psychological Association (1974), "the merit of a criterion-related 
validity study depends on the appropriateness and quality of the 
criterion measures chosen" .(p. 27). Each of the seven criterion 
measures will be discussed in terms of its reliability and validity. 

Test of Written language (TOWL) . The reliability of the TOWL 
(Hammill & Larsen, 1978) was determined using measures of internal 
consistency, i.e., the average correlation among all the items in a 
test (Nunnally, 1978), test-retest, and inter-scorer reliability. The 
internal consistency coefficients ranged from .77 to .89, t€»st-retest 
coefficients ranged from. .41 to .90, and inter-scorer rel iabil ity 
coefficients ranged from .76 to .98. The authors caution the examiner 
about interpreting scores on subtests with reliability coefficients 
below .80. 

The validity of the TOWL (Hammill & Larsen, 1978) was determinod 
through criterion-related validity correlations with Myklebust's 
(1965), PSLT. Content validity was discussed by reporting an item 
analysis of the test, and construct validity was assessed with three 
measures. The many reported validity coefficients ranged from .10 to 
.81. Essentially, this large range of coefficients forces the 
judgment of the TOWL's validity on the examiner. Salvia and Ysseldyke 

a 
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(1978) concDr that judgment of the validity of the content of a' test 
is .often a matter of expert opinion, and that achievement tes.ts mus\ 
reflect the content of the curriculum if they are to provide relevant\ 
information. 

For the present study, the raw score total from the TOWL was 

/ „ 

utilized as. a criterion measure, with the adjustment used by Oeno, 
Mirkini and Marston (1980): 

The TOWL consists of five subtests: Vocabulary, Thematic 
Maturity, Spelling, Word Usage, and Style. The criterion 
measures obtained from t'he TOWL were the raw scores from the 
five subtests and the raw total of the five subtests. It 
. should be noted that four of the five subtests are 25-.item 
scales, and therefore were equally weighted in the raw 
total. However, the Vocabulary Subtest does not have a 
limited number of items and as a result students may obtain 
a score higher than ?5 points on this subtest. We found the 
range of Vocabulary scores in our sample of 135 students to 
be from zero ,r> 70 points. " To ensure that the Vocabulary 
subtest was equally weighted in the raw score total, a 
student's scort on this measure was multipli'ed by a 
correction factor of .357 (or 25 -=- 70). (Oeno, Mirkin, & 
Marston, 1980, p. 13) 

Developmental Sentence Scoring . Lee and Canter's (1971) 
instrument, which measures syntactic maturity, also was considered to 

be a criterion for establishing concurrent validity. The validity of 

\ 

this instrument can be inferred from a study conducted by Rubin,. 
Buium, and Balow (1975), which indicated that a general correspondence 
existed between early verbal grammatical forms and levels of > words 
produced most frequently in writing. • \ 

Originally developed to describe syntax in oral language. 
Developmental Sentence Scoring (Lee & Canter, 1971) was used in, the 
present study as a criterion measure of the syntactic maturity of Ithe . 
written samples. Appendix A outlines the scoring categories adopted 
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from this approach. 

Mean T-unit Length . Hunt's (1966) mean T-unit length has been 
accepted as a reliable measure or syntactic complexity (Dixon, 1972; 
Hunt, 1977). It also is considered to be a valid measure since it 
distinguishes between students' writings (Dixon, 1972).. 

Hunt's (1966) rules for st:oring the mean T-unit length in a 
wrftten composition were utilized in the present study. An example of 
the mean T-unit length scoring procedures are presented in Appendix B. 

Checklist of Written Expression . The reliability of Poteet's 
(1980) checklist was determined by having six scorers complete the 
checklist on samples of written expression. An interscorer percent of 
agreement was obtained for all six scorers as well as for two sets of 
p.airs of scorers. Inter-score agreement for the six scorers was 70.5%, 
and the average inter-score agreement for the pairs was 47.7%. 

The validity of Poteet's Checklist of Written Expression as yet 
has been determined only by its face validity, i.e., the extent to 
which an instrument "looks like" it measures what it is intended to 
measure (Nunnally, 1978). , According to reseachers (Deno, Mirkin, 
Lowry, & Kuelinle, 1980: Hammill & Bartel, 1978; Wallace & Larsen, 
1978), a measure of written language should include penmanship, 
spelling, grammar, and ideation, all of, which are elements of Poteet's 

checkl ist. ' . ^ 

> 

Poteet . (1980) developed his checklist to be an informal 
assessment of written expression. Its four components. Penmanship, 
Spelling, Grammar, and Ideation, are each divided into s^ub-components. 
A check mark is placed in one of the four columns. Too Advanced (TA), 



Adequately *Used (A), needs to.be Introduced (1), and needs Remediation 
(R), that best describes the student's achievement in that component 
of written expression. 

For purposes of this study, each student's sample was rated only 
as Adequate or not Adequate. The following components and sub- 
components of the checklist were disregarded due to their 
inappropriateness for the present study: Pencil Grip, Spelling sub- 
components A-J, Syntax, Product ivity„^nd Word Choice. Syntax was the 
only entire component that was disr^egarded. Preliminary scoring 
indicated that the Syntax component and sub-components were too 
subjective for inclusion iu the present study. Productivity was, 
disregarded because the total number of words that were written was 
included in the Total Words Written measure. Hermreck (1979), in her 
study with the preliminary checklist, stated that in Poteet's informal 
assessment, the examiner is free to delete parts since no totaTscores 
are derived from the checklist. 

Holistic Rating Scale . The holistic rating scale was completed 
by two scorers. The inter-scorer percent of agreement was 2^% for 
perfect agreement, and 68% for agreement within plus or minus one 
number on the rating scale. 

Cooper (1'977) stated that holistic rating scales are a "valid and 
direct means of rank ordering students by writing ability" (p. 3). 
Based on the inter-scorer agreement on the holistic rating scale 
utilvized in the present study, and Cooper^s (1977) statement, use of 
this scale appeared appropriate for purposes of this study. 

Raters were asked to read each sample, and give ratings cf their 



over-all Impression of the quality on a scale from 1 to 7. A score of 
1 was poor, 4 was average, and 7 was excellent. A copy of the raters' 
written instructions is provided in Appendix C. 

Wordf. Spelled Correctly and Total Words Written . The reliability 
of scoring the number of words spelled correctly, and the total number 
of words written in a student's composition, was assessed by Marston 
and Deno (1981). They determined that "all Tieasures appeared to meet 
the professional standards set for reliability" (p. 19). The test- 
retest reliability coefficients ranged from .62 to .81, for Words 
Spelled Correctly, and from .64 to .91 for Total Words Written. 
Internal . consistency coefficients ranged from .70 to ^.97 for Words 
Spelled Correctly, and from .87 to .99 for Total Words Written. The 
mean inter-judge reliability of these two. criterion measures seemed 
quite adequate. 

The validity of counting the number of words spelled correctly 
and the number of total words written was determined by Deno, Mirkin, 
and Marston (1980). Correlations with the criterion measures range 
from .56 to .71 for Words Spelled Correctly, and from .56 to .72 for 
Total Words Written. Deno, Mirkin, and Marston (1980) stated that 
correlational data from the combined samples substantiated the 
validity of total Words Written and Words Spelled. Correctly as 
measures of written .expression. 

For Words Spelled Correctly, incorrectly spelled words were 
checked, and the remaining words were counted to obtain the total 
number of correctly spelled words in each written sample. An exarrple 
of this procedure is proyided in Appendix 0. » * 
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In the measure of Total Words Written, the words in each written 
sample were counted, whether or not they were spelled correctly. 
Except for single letter words such as "I" and "a," two or more 
letters in sequence were counted as words. An example of this 
procedure may be found in Appendix E. 
Scorers 

Six teachers rated student samples according to Poteef's (1980) 
Checklist of Written Expression. Four of these teachers held Master's 
degrees and five were certified in regular and special education. All 
six teachers had at leas£ four years of teaching experience; all were 
employed by the same school district. 

Two teachers rated the 50 written samples on the holistic rating 
scale. On^ teacher held a Master's degree and had 20 years of 
teaching exper)1ence, and the second teacher held a Bachelor's- degree 
and had four years of teaching experience. 

One teacher and one non-teacher scored correct word sequences. 
The teacher was completing her Master's degree, and had four years of 
teaching exper^nce. The non-teacher had completed eight years of 
college and was practicing as a Doctor of Dental Surgery. 
Procedure 

Three trained graduate research assistants tested the students on 
an individual basis. Jesting required about 55 minutes per student. 
Students were asked to write for five minutes in resppnse to a Story 
Starter o/Topic Sentence (Oeno, Mirkin, & Marston, 1980), Since the 
correspondence between Story Starters and Topic Sentences for use as 
stimuli for written expression has been adequately demonstrated (Deno, 
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Mirkin, K Marstori, 1980), student samples based on either of the two 

stimulus methods were acceptable for the present evaluation of written 

expression • A copy of a Story Starter and a Topic Sentence, as well 

as a list of all Story Starters and Topic Sentences utilized in the 

present study, may be found in Appendix F» 

After each child was presented with a Story Starter or Topic 

Sentence, the examiner recited the following instructions: 

. I want you to write another story. I am going to read a 
sentence to you first, and then I want you to write a short 
story about what happens. You will have a minute to think 
about the story you will write and then have five minutes to 
write it. "When I say 'please start writing,' you may begin. 
(Deno, Mirkin, & Marston, 1980, p. 1*6). 

All students were allowed five minutes to complete their compositions. 

The tOWL (Hammill & Larsen, 1978^) also was administered to the 
students. Administration procedures in the instructional manual were 
followed; administration of the test took approximately 40 minutes. 

Each composition was later scored using each of the remaining 
criterion measures (Developmental Sentence Scoring, Hunt's mean T-unit 
length, the Checklist of Written Expression, the hplistic rating 
scale. Words Spelled Correctly, ^ and Total Words Written). In 
addition, the written samples were scored for the number of correct 
word sequences. Correct worfl sequences were defined as two adjacent, 
correctly spelled words that are acceptable within the context of the 
phrase to a native speaker of the English language. The term 
"acceptable" means that a native speaker would judge the word 
sequences as syntactically and semantically appropriate. The caret 
method (White & Haring, 1976) for scoring correct and incorrect word 
sequences was utilized. 
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A portion on the scoring procedures for Correct Word Sequences 
follows: 

This method involves placing a "caret" ( ^ ) over every 
correct , sequence, and a "oaMtl' (v) under every incorrect 
sequence. Examples are pr(m€ecKb^ Note that the first 
sequence is considered "bfaik-^^ (a sensible 

^ start)', ard "last-word-tyblarHT^I^^ This 
means that there v/ill ajjwdys be one more sequence than words 
» in the' phrase. y 

Example 1:^ The'^b^ll high'' in'' the aire. 

1 V V V 

Seven sequences ans^poss.ible, -fogr are correct and three are 
incorrect. \ 

In example I, the carets at the beginning and at the 
end of the sentence are for a sensible start and end. The 
remaining correct carets denote correct word sequences. The 
incorrect caret^s in the example denote word sequences that 
^ are not correct. 

An omission of one or more words is possible, but the 
important aspect is that "ball high" does not constitute a 
correct sequence in that sentence. Since "aire" is an 
incorrect sequence because it is spelled incorrectly, count 
incorrect carets before and after the misspelled word. With 
a misspelled word, the sequences on both sides of the 
incorrectly spelVed word are incorrect. 

The examiner received approximately 15 minutes of training, which 

f 

consisted of explaining the scoring procedures for counting correct 
word sequences. A copy of the complete scoring procedures may be 
found in Appendix G; an example of this measure is provided in 
Appendix H. Scores were obtained by counting the total number of 
correct word sequences and the total number of incorrect word 
sequences. In addition, the number of minutes required to score each 
student sample was tabulated. 

Each of the six teachers scoring Poteet's (1980) checklist were 
provided with a 30-minute group training session, as well as printed 
instructions covering the scoring procedures of the checklist. They 
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were instructed to work independently on their packets of written 
expression samples and checklists. Copies of the scoring procedures 
and the Checklist of Written Expression (Poteet, 1980) are provided in 
Appendices I and J, respectively. 

Two scorers rated each written sample according to Cooper's 
(1977) .General Impression Marking procedures for the holistic rating 
scale. Each rater, received approximately five minutes of training as 
well as the printed procedures. 

Results 

The first question examined whether elementary students* samples 
of written expression could be scored consistently in terms of the 
number of correct word sequences. Two preliminary sets of 10 samples 
of written expression were scored by three scorers. The scorers marked 
the first set of samples disregarding spelling errors. This meant that 
a word that was spelled incorrectly, but was still adequate in terms 
of grammar, 'capitalization, surrounding punctuation marks, and 
content, was counted as correct. Each correct and incorrect word 
sequence was tabulated for agreement between each of the 10 pairs of 
samples in order to obtain the percent of agreement for the word 
sequences in those samples; the average percent agreement was 85.8?J. 

Since spell ing has been determined to be an important aspect of 
written expression (Deno, Mirkin, Lowry, & Kuehnle, 1980; }1ammill & 
Bartell, 1978; Page, 1968; Poteet, 1979; Slotnick, 1973; Wallace & 
Larsen, 1978), and since the scorers general ly demonstrated less 
agreement on incorrectly spelled woirds, the scoring procedures were 
altered so that a misspelled word would ba scored as an ^incorrect word^ 

I? 
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sequence. The 10 samples then were rescored, with spelling errors 
counted as incorrect word sequences. Results showed the percent of 
agreement .to be 90. 5X, almost a 5% increase over the earlier 
preliminary percent of agreement. 

Utilizing the revised scoring procedures, 20 samp^les of written 
expression were scored by two scorers. Percent of agreement on these 
samples was 90. 3X. 

The second question examined the number of correct word sequences 
that could be considered as typic?! fbr elementary students in grades 
3-6. The mean number of correct and incorrect word sequences per 
grade level is provided in Table l\ 



Insert Table 2 about here 

\ 

As seen in Table 2, the number of correct word sequences ' 
increased as the grade level increased. Sixth graders wrote more than 
twice as many correct word sequences as the third graders. In 
contrast, the number of incorrect word sequences was relatively 
consistent -across grades 3-6. 

The final question of this study examined the relationship 
between the number of correct word sequences and the criterion 
measures. Correlations between correct and incorrect word sequences, 
and each of the seven criterion measures of written expression are 
presented in Table 3. , 
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Insert Table 3 about here 



Investigation of Table 3 reveals that the highest correlations 
with Correct Word Sequences were found for Words Spelled Correctly 
(•92), Total Words Written (.91), the Holistic Rating (.sHit^and the 
raw score total on the JOWL (.69). A moderate correlation wasH^nd 
between Correct Word Sequences and Developmental Sentence Scoring 
(.49). All of these correlations were significant. at the .001 level. 
Mean T-unit length and the Checklist of Written Expression subtest 
scores generally were not associated with the number of correct word 
sequences. 

As seen by inspection of the correlatidns with Incorrect Word 
Sequences, a moderate negative correlation was found for Poteet's 
Capitalization subtest (-.46). A negative correlation also was 
obtained between the raw score total on the TOWL and Incorrect Word 
Sequences (-.23); ' 

To further demonstrate the relationships between word sequences 
and the criterion measures, an intercorrelation matrix is presented in 
Table 4. As can be seen by close examination of Table 4,, Correct Word. 
Sequences was virtually unrelated to Poteet's (1980) Checklist of 
Written Expression subtests. However, only one other criterion 
measure. Developmental Sentenqe Scoring, correlated with the 
Capitalization subtest and that correlation was low (.28). The other 
criterion measure that exhibited a low correlation with Correct Word 
Sequences was Mean T-unit Length. Again, however, only Total Words 



16 . 

Written and words Spellecl Correctly correlated (.39 and .38, 
respectively) with Mean T-unit length. In essence, the two criterion 
measures that had low correlations with Correct Word Sequences also 
had low correlations with the other criterion measures of written 
expression utilized in the present study. All of the criterion 
measures that correlated significantly with Correct Word Sequences 
also correlated with the other criterion measures. As evidenced in 
Table 4, the two criterion measures that had significant negative 
correlations with Incorrect Word Sequences--the Capitalization 
subtest, and the raw score total on the TOWL--correlated positively 
with other criterion measures utilized in the present study. 

Discussion 

The purpose of the study presented here was to determine whether 
Correct Word Sequences is a valid measure of written expression that 
could qualify for use in a formative evaluation system for e.lementary 
students in grades 3-6^ Three questions were posed to examine the 
validity and potential usefulness of counting correct word sequences 
as part of a formative evaluation system. 

Results related to the first question (i.e., determining the 
consistency with which samples of written expression could be scored) 
indicated a high percentage of agreement between two /scorers. This 
finding provides evidence for the reliability of counting correct word 
sequences, utilizing the revised scoring procedures, for measuring 
students' written expression sampTes. 

Typical performance for elementary students, grades 3-6, in 
producing correct word sequences during a five minute writing sample 



was the second question exam^irfed. The scores ranged from 27.3 for 
third graders to 58.8 for sixth graders. This finding is important 
since it indicates that Correct Word Sequences measures improvement in 
the written expression of elementary students. The number of correct 
word, sequences increased by approximately 10 with each successive 
grade level. 

Examination of the relationships between Correct Word Sequences 
and the seven criterion measures was the final question of the present 
study. Correct Word Sequences was found to correlate significantly 
with five of the seven criterion measures used in the present study: 
the Holistic Rating Scale, Total Words Written, Words Spelled 
Correctly, the raw total on the TOWL, and Developmental Sentence 
Scoring. The correlations indicated that counting the number of 
correct word sequences is a valid measure of elementary students' 
written expression in grades 3-6. It is important to note the high 
correlation between Correct Word Sequences and the Holistic Rating 
Scale. This correlation indicates tliat Correct Word Sequences 
strongly represents what is considered as appropriate written 
expression. ' * 

Interestingly, Total Words Written and Correct Word Sequences 
both correlated with the Holistic Rating Scale at levels considerably 
higher than did the tOWL, Therefore, both Total Words Written and 
Correct Word Sequences appear to be superior to the TOWL in measuring 
what generally impresses people as good written expression. In 
addition. Correct Word Sequences significantly correlated with other 
criterion measures at a higher level than the TOWL correlated with the 
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other criterion measures. This finding raises some questions about 
the validity of the TOWL. 

The extremely high correlations between Correct Word Sequences 
and Total Words Written, and with Words Spelled Correctly, also help 
to substantiate the validity of counting correct word sequences to 
measure written expression. Total Words Written has been shown to be 
a valid indicator of written language ability (Brigham, Graubard, & 
Stans, 1972; Oeno, Marston, & Mirkin, 1982; Hunt, 1966; . Myklebust, 
1965), as well as Words Spelled Correctly (Oeno, Marston, & Mirkin, 
1982; Page, 1968; Slotnick, 1973). The correlation of these measures 
with Correct Word Sequences is important due to the similarity of all 
three measures. Counting the number of correct word "sequences 
includes the total number of words written and spelled correc,tly; 
however, the difference is in the stipulation of appropriate semantics 
for Correct Word Sequences. 

i 

A moderate correlation existed between Correct Word Sequences and 
Developmental Sentence Scoring, indicating that counting the number of 
correct word sequences can moderately represent the results of the 
complex scoring system of Developmental Sentence Scoring. 

Incorrect Word Sequences, while moderately correlating negatively 
with two criterion measures, exhibited the interesting property of 
relative consistency across grade levels 3-6. It appe.ar^ that 
although the students* writing improved with each grade level, the 
number of incorrect word sequences remained rel,atively unchanged. It 
seems expected that as the number of correct word sequences increases, 
the number of incorrect word sequences would decrease. Improvement is 
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generally associated with fewer errors. However, the results of the 
.present study indicated that this may not be the case. As elementary 
students' written expression improves, they are writing more .correct 
word sequences, rather than lowering their number of incorrect word 
sequences. 

A final comment should be made about efficiency in scoring 
Correct Word Sequences. The mean length of time required for scoring 
the samples was 4.3 minutes, with this time less for the shorter third 
grade samples and greater for the sixth grade samples. Thus, time 
.requirements are not great, but the amount of time for scoring is 
substantially greater than for total words. 
Conclusio n 

The results of the research p/esented here indicate that counting 
the number of correct word sequences is a valid and reliable measure 
of written expression for elementary students in grades 3-6. Scoring 
procedures resulted in high agreement t)etween two scorers, and typical 
performance of elementary students in grades 3-6 showed expected 
increases^ with grade level. Correct Word Sequences appears to meet 
most of the criteria necessary •to qualify for use in a formative 
evaluation system. 

A need for further research utilizing Correct Word Sequences is 
obvious. Empirical data on the- use of Correct Word Sequences as a 
measure of written expression in an actual formative evaluation system 
is imperative* .The extent to which growth is visible through daily 
measurement needs to be de.termined. Further, whether parents and 
students can be easily taught to administer and score this measure 
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consistently and accurately need.; to be documented. 

Counting the number of correct word sequences may not appear to 
be as impressive as other measures of written expression because it is 
not a published measurement device requiring complex scoring 
procedures. However, Correct Word Sequences has been determined to be 
highly representative of what is considered as appropriate writing, 
and show^ high, correlations with other valid measures%f written 
expression. This is not true of many of the more elaborate measures 
of written expression. 
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Table 1 

Characteristics of Students Whose Written Expression 
Samples were Scored for Word Sequences 



Grade 


N 


Males 


Fejnales 


towl" , 

Raw Total 


3 


.' 9 . 


5 


4 


68.8 


4 


20 


7 


13 


80.^ 


5 


11 


4 


7 


92.8 


6 


10 


3 


7 


94.7 



0^ 



Table 2 

Means and Standard Deviations of Number of Correct and Incorrect 
^ * Word Sequences at Each Grade 





N 


Mean 


Standard Deviation 


Correct Word Sequences 








Grade 3 


9 


27.3 


13.9 


Grade 4 


20 


41 .3 


24.7 


Grade 5 


11 


48.2 


26.2 


Grade 6 


10 


58.8 


27.2 


Incorrect Word Sequences 








Grade 3 


9 


15.2 


5.7 


Grade 4 


20 


13.7 


10.7 


Grade 5 


11 


16.3 


7.3 


Grade 6 


10 


12.6 


6.2 




2a 



Table 3 

Correlations Between Word Sequences and Criterion Measures 





Raw Total 


Hoi istic 
Rating 


Poteet^ 
Pen. Cap. 


Pun. 


Developmental 
Sentence 
Scoring 


Mean 
T-unit 
Length 


Total 
Words 
Written 


Words 
Spelled 
Correctly 


Correct ^ 
Word . 
Sequences, 


,69** 


.85** 


-.03 .20' 


-.06 


,49** 


.18 


.91** 


.92** 


Incorrect 
Word 

Sequences 


r 

-.23* - 


-.02' 


.14 -.46* 


-.26 


.01 


,22 


.11 


.06 



^Significant correlations are indicated by * {£ = .05) and ** {£ = .001). N = 50. 

'^The components of Poteet's checklist included were Penmanship (Pen.), Capital ization ( Cap. ) , and^ 
Punctuation (Pun.). 



Tab] 9 4 
Intercorrelatlon flstrlx* 





correct incorrect TOWL 
Word Word Raw Holistic 
Sequences Sequences Total Rating 


Poteet ^ 
Pen. Cup. Pun. 


Developmental 
Sentence 

Scor i no 


Mean 
T-unit 
^Length 


Total 
Words 
Wri tten 


Words 
Spelled 

CorrAC t1 V 


Correct Word 
Sequences 


-.21 ,.69** .85*** 


-.03 .20 -.06 ^ 


.49** 


.18 


.91** 


;92** , 


Incorrect Word 
Sequences 


-.23* -.02 


.14 -.46** -.26 


.01 


. .22 


.11 


.06 


TOWL - Raw 
Total 


.61** 


.08 .09 -.08 


.31* 


.07 


.66** 


.68** 






.10 .15 -.16 


.45* 


-.21 


.85** 


.84** 


Poteet - ten; 




- - -.10 -.25* 


.06 


-.09 


.00 ^ 


-.01 ' 


Poteet - Cap. 


* 


.17 


.28* 


.19 


.09 




Poteet - Pun. 






-.06 


.09 


-.11 


-.10 


Developmental 
Sentence 
Scoring 


* 






.13 


.51** 


.52** 


Mean T-unIt 
Length 










.39* 


.38* 


Total Words 
Written 












.99* 


Words Spelled 
Correctly 














•significant correlations are indicated by *(£ ■ .05) and 


**(£ ■ .001). N « 50. 










The components of Poteefs checklist included were Penmanship (Pen.). Capitalization (Cap.), and Punctuation (Pun.). - 
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APPENDIX A 



Category 

Indftfinlct Pronouns 
or Noun Nodifitrs 



Score 

1 
2 
3 
4 
5 



Criteria . 

it, this, that 

no, some, more, ''sail, etc. 

something , somebody , someone 

nothing, nobody, no one, none 

any, anything, anybody, anyone, 
every., etc. 

both, feV| many, each, several, 
most,^ etc. 



Personal Pronouns 



1 

2 
3 
4 
5 

6 

7 



Isc and 2nd person (I, me, you, etc.) 
3rd person (he, him, his, she, etc.) 
plural pronouns (we, us» chey, etc.) 
those, these 

reflexive pronouns (myself, yourself, 
etc.) 

wh"pronouns (who, which, etc.); 
wh-word + inf'initive 

(his) own, one, oneself, whichever, etc. 



Main Verbs 



1 
2 
3 

4 
5 



7 
8 



uninflected verb; copula ;(is, 's) 
is + verb + ing 

*-s, *-ed; irregular past; copula am, 
are, was, were; auxiliary am, are, 
was, were 

can, will, may + verb; obligatory do 
+ verb; emphatic do + verb 

cou^d, woulci, should, might verb; 
^ obligatory does, did + verb; 
emphatic does, did + verb 

must, shall '+ verb; have + verb + 
en: have ('ve) got 

passive, any tense 

have (had) been + verb + ing; modal 
+ have + verb + en; modal + be 
+ verb + ing; other auxiliary 
combinations 



. c 



Cattgory 
Secondary Verbs 



Score 

1 
2 
3 
4 



5 
6 



Criteria i 

early developing, infinitive complements 

noncomplementing infinitives 

participle, present or past 

early infinitival complements with 
differing subjects in kemals; 
later infinitival* complements; 
obligatory deletions; infinitive 
with wh-word 

passive infinitival complement 
gerund 



Negatives 



itt thlsi that + copula or auxiliary 
is, 's + not 

can't, don't 

isn't, won't 

copula - negative or auxiliary- 
negative contractions; pronoun- 
auxiliary contraction + not; un- 
contracted negatives 

negatives with have; auxiliary have- 
negative contraction; pronoun- 
auxiliary have contraction 



Conj unction 



1 
2 
3 
4 
5 
6 



and 

but. 

because 

so,' and so, so that, if 
or, except, only 

where, when, for, till, since, as, 
i etc. + . adjective + as, as if, etc.; 
obligatory deletions; optional 
dele^ous; wh-words + infinitive 

therefore, however, 'whenever, etc. 



A.3 



Category 



Score 



Criteria 



Interrogative Rcversala 1 

2 
3 



reversal of copula 

reversal of auxiliary be 

obligatory do, does, did; reversal of 
modal; tag question 

reversal of auxiliary have; reversal 
with any two auxiliaries 

reversal of three auxiliaries 



Wh-Queations 



1. 
2 
3 
4 



who, what, what + noun 

where, how many,' how much, etc. 

when, how, how + adjective 

why, what if, how come, how about + 
gerund 

whose, vhich, which + noun 



' r 



37 



APIENDIX B 
Example of T-Unlt Scoring 



^WhOT I ymnt on a trip to Texas I rode on an alrplan^ ^ had a 
really good tlae^^ stayed In a hotel for three day^^d I went 
to a lot of parks^ 



T-»unlts ■ 4 
Words - 34 

Mean T-unlt Length - 8.5 
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APIENpIX C 
Instructions for Scoring Samples with 
the Holistic Rating Scale 



(1) Read the sample. 

(2) Please rate your over-all impression of the quality 
of the student's written sample. 

(3) The scale to rate each sample on is as follows: 

1-2.3-4-5-6-7 
poor * average excellent 

(4) Put the rating beside the appropriate number of the 
sample . 



A- 
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APPENDIX D 

Example of Words Spelled Correctly Scored 



When I vent on a trip to Teksas I road on an aerplane. I 

✓ ✓ 

had a reely good time. I staid In a hotel for 3 days and I vent 

to a lot of parks. 



Words Spelled Cotfrectly - 28 



0 \ 
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APPENDIX E 



£»nple of Total Words Written Scored 

When I vent on « trip to Texas I rode on an airplane. I 
had a really good tine. I stayed in a hotel for 3 days and. 
I vent to a lot of parks. , 



.Total Words Written « 33 



Foro B^l Story Starter ' APPENDIX F , 



a stor/ about tht nighfyou were camping in tht wood^ arid you heard 
strangt noises in the woods ^ 



;^ERic 



TV MWCl >rwC2- BWl BW2 TUX ■ TL'2 SPl 



F-2 

row B-2 Topic S«nc*t)ic* Itgae 



_ I~|int th»t you could travel *nyvh«." in th.'vo^ld" thac you' wanted. 
Where would you go? Why? — 



U tWCl MyC2 3V1 BW2 TUl TU2 SPl ^^tTT^ 

□ □ □ □ □ DDp C 
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List of all the St^ry Starters 
and Topic Sentences Used 



Story Starters 

!♦ Tell a story about what'lcind of car you would buy 
and what features it would have. 

2. Pretend you are stranded on a tropical island by 
yourself. Tell a story about what happens to ybu* 

3. Wirte a story that ^begins with: One summer I went 

on a trip. - 

» »* ♦ 

if. Write a story that regins with: One day something 
happened which made me very happy. 

5. Tell a story about the night you were camping in 
the" woods and you heard strange noises in the woods 

6. Write a story that begins with: One night I went 
outside when ."it was very dark^ 

7. Pretend that you can travel anywhere that you want. 
Where would you go? How would you get. there and 
what Would you do when you got there? 

Topic Sentences 

1. Describe your favorite season of the year and tell 
why* / 

2. What is your favorite Holiday of the year? Give 
your reasons. 

3« .Imagine you could travel anywhere in the world that 
you wanted. Where would you go? Why? 
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APPENDIX G 
Scoring Procedures 
Read the entire sample . 

Based on the context of each phrase, score each written 
sample according to Correct Word Sequences * Correct Word 
Sequences are two adjacent, correctly spelled words that are 
acceptable within the context of the phrase to a native 
speaker of the English language. The term "acceptable" means 
that a native speaker would judge the word sequence as 
syntactically and semantically appropriate. 

The caret method for scoring correct &nd incorrect word 
sequences will be used. This method involves placing a "caret" 
(^) over every correct sequence, and a "caret" (v) under every 
incorrect sequence. Examples are provided below. Note that 
the first sequence is considered "blank-to-first-word" (a 
sensible start), and "last-letter-to-blank" (a sensible end). 
This means that there will always be one more sequence than 
words in the phrase. 

Example 1 1 ^ The^ball high'^in^the aire . 

Seven sequences are possible, four are correct and three are 
incorrect. 

A A A A A A 

Example, II ; The cats, sits drinks milk and she likes to play 

A A A 

her name is Sal I. like her to. 

V V/ V V V V 

Twenty sequences are possible, ten are correct and ten are 
incorrect . 

In Example I, the carets at the beginning and at the end 
of the sentence are for a sensible start and end. The remaining 
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correct carets denote correct word sequences. The incorrect 
carets in Example I denotes word sequences which are not 
c orre c t • 

An omission of one or more words is possible, but the 
important aspect is that "ball high" does not constitute a 
correct sequence in that sentence. Since "aire" is an in- 
correct sequence because it is spelled incorrectly, count 
incorrect carets before and after the misspelled word. With 
a misspelled word, the sequences on both sides of the, incor- 
rectly spelled word are not correct. 

In Example II, each caret will be explained as follows: 
^ The : a sensible start for that phrase (sentence). 

The cats: based on the rest of the sentence, cats should 
be ca>t , - - - ■ 

cats/Sits: / should be cats sit or c^t sits, .not cats^sits, 

sits drinks > omission of at least one word, 

drinks'^milk - to piay> correct word sequences. 

Sal I: The period was inappropriately placed, which makes 
^ the sequence not sensible, 

"I.: sensible phrases do not end with I. 

, like: this phrase should riot start with like , Since 
^ it comes after a period, it .should have been 

capitalized-; therefore, it is misspelled and 
marked accordingly, 

like her to,: like her is incorrect because like is mis- 
^ ^ ^ spelled. Her to is incorrect because to 
is misspelled. To, is incorrect because 
it is misspelled. 

Any compound words that are written seperately should be 
counted as one word that has been misspelled. 

If a period or comma is missing where there obviously 
should be one, count* that sequence as incorrect. 



If there is not a period at the end of the written sample, 
do not count the last word as correct or incorrect because the 
samples were timed; the student may not have .had time to, finish 
the sentence . , 

When a student copies instructions or words are unread- 
able, skip those words. Do not count as incorrect, and start 
counting carets with the first two words that are readable and 

. K 

not instructions. -v 
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torn B-I Story Starter 



APPENDIX H 



Name 



070323000001402010 



Tell a story about the niiht-you were camping in the woods and you heard 
strange noises in the woods. 

A 



/\ + ^-4- ' ' 0-"" \ A 




J? 



16 ; <7 V 



3ir 



■ir - - ^ 



1 

Ll^2 uiA^ 





1? 




6) 


0 


2 


n 


Co 










1 


\ 


-i 


^3 






■ I 




1/ 




/ 








S.I 1 


%- 








/ 














E] 0 


- 1 




:i u'2 

'■| \A 


•jri 
I - : 


T" f > 1 
• »■ ■ 


3 
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APPENDIX I * 
tfritt^n Exprestion Checklist 



I. Penmanship 

1. Rating - Circle 1 2 3 ^ 5 as appropriate. • 

the first paijagraph is indented. 

B. Spacing, ofthescniences: Mark according to whether or not there 
Is adequate space left between each sentence. 

C. ST,P,cin.? of the words : Mark according to whether or not there is 
adequate space lel't between each word. 

D. Spacing of letters ! Mark according to whether or not there is 
adequate space, lei't between each, letter. 

E. Slant : Mark according to the adequacy of the general slant of 
the letters. / ' 4.^ 

- " F. T/^ ttbr formations : Mark according tp whethe,i/or not the le;v,c.r 
■formations are adequate , -not perfect. ' ^ ■ .1 

r Presc-ur-e on the tjacer: The lightness or darkness of the letters 
°' irf g- indicator OX' t oo little, or too much. pressure on the paper. 

H. Skip 

:i. Spelling ^ , . -tt^A 

< misspelled. Please complete -according to:numl:er of misspelled 
' words d ivided by the total number of words. , . 

Place a tallv mark, one for every misspelled word, under the 
section fitted Notef if-there -aFn^o. errors in.one word, then 
!!rv ?hfi tallv k§H^ding to only the first error in the word, tor 
TxlLlT. S ihe?e are tfo misspelled ^?57di. and one of those words 
tos two errors, there should still be only two tally marks (.-v-G) 
in the spelling section. 



III. Grammar ' , / A 

A Cap+ialiration: (Do 1-11? skip 12 Place a check (v^) marK 
according to W hether or not the student C3.pitalized those , types 
of Sords adequately or inadequately. If there is one woro 
correctly capitalized, and one word -incorrectly capitalized, useyoi 
judgement as^.o whether or not you feel the student knows the 
correct way of capitalizing those types of words. If the student 
SiS nat us^ a-typi of capitalization rule . and the sample is 
■till adequately. written, then do not check anything. 
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\ Page 2 

B. Punc tu ation I (Do 1-11 1 skip 12) Put a chock mar^ according 
to whether or, not the studt^nt used ench type of punctuation 
mark adequately or inadequately. If the student' did n^t use a 
type of punctuation mark that ahould have been used, ch6^k ad^uate. 
or inadequate based on your judgeinent of v/hexher or not you ff * 
the student knows .how to use that type of punctuation corrflcx^^. 
If the student did not use sons of the different typrs of' ^ 
punctqatlon marks, and these types were not needed, then leave 
those spaces on checklist blank. 

Syntax i Skip . 

IV. Ideation 

A. T:n;>e of writing ! Please check one of the five choices. 
E* Substance I 

" 1. Naning - writer lists objects or actions. Example » I see 
a dog* 

2. Descripl^ion - objective reporting. Example i The dog sat 

dovm. c ' 

3* Plot - a completed (or somewhat completed) story hossMLver 

short or long. Remerfitfer that these saiwples were timed and 

the student may not have had time to complete his story. 

Issue - beyo^nd a simple plot? it deals with spjie moral ther.e 
, »or - issue . / * ' ' ' 

C« Productivity t Skip. 

D. Comprehensibility : Self rexplanatory . 

\* Reality : Self-explAnatory* 

p; style I 

i • Sentence Senses For each sentence in the sample (use the 
sentences the student wrote that end in a period. Do not 
mark adlsording to where the sentences should be.), mark one 
tally in one of the three j>ections of Completeness . 

a) Completeness - put a tally mark according to whether 
t each sentence is cpmplete (subject and predicate), 

run-on, or just fragments of a sentence. 

Only use those sentences you marked as being complete, not ^ 
run-cn or 'fragr.onts for tally marks under structure and typo 

b) Structure - put a talJLy mark according to whether each 
sentence is simple, compound (two sentences combined 
with and ) , complex (use of prepositional phrases set 
off by commas, use of but , v/'hich , etc.), or compound 
and complex. 

c) Types - put a tally mark accordirgto whether each 
sentence is declarative (.)» interrogative (?) , 
imperativa (. but much stronger than declarative), 
or exclamatory (! ). 

2. Tone I Described, in relationship to the di'stance the writer* 
establishes between hincelf/herself and the reader • 

3. Word Choice \ Skip. 
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PENMANSHIP!^ 

Rating: 12 3 4 5 

A. Spacing on -the page 

B. Spacing of the aeniencc$. 

C. Spacing of the words 

D. Spacing of letters 
E . Slant 

"F. Leiicf formations 

Pressure on the paper. 




II, SPELLING ^ 



Av Miscalled ralc_ 

Bv Leuer insertion 

C Letter omtsslon 

D Le^er substuiriion 

E. Phonetic speDing 

F. Directional conKision. 



G. ScKwAorr-conrrolled voweU... 

H Lcmr orientHiion 

L Seqoence^ 

J Other 
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111. GRAMMAR 

A. CopHQlitation \^ 



proper noun 
proper adjective 

first word in a sentence 



?. exclamation mark 

8 colon 

9 ihe dash 

10. parentheses 

11. brackets 
\t the slash 

f^i^s of speech 

a. V( 

b. nouns 

c. pronouns 1 

d. adjectives 

e. adverbs 

f. prepositions 

g. conjunctions 

h. interjtsctions 
agreement 




first word in a line of verse 
first word In a quotation 
6, principal words in a title 

7 personal title 

8 use of or "O", 

9 personification 
10 salutation in a letter. 

11. complimentary close tn h tetter 

12, other. 

B- PunclUGf/on 1^ 

1 period 

2 comma 
- 3*~ aposlrophe-i: 

4 quotation marks 

5 question mark 

6 semicolon 



3. case 

4 pronoun reference 

5. order/.position 

6. paralletisi 

7. abhi^^wTons/numbers 



J-3 



\ 

IV. IDEATION 



A. Type o/ wm^ \ 
1, $tory_: 2. poem 3. letter 4. report S. rcvie\V 

1. Naming 2. DescripUon 3. Plot 4. Issue ^ 



C. <>uiitw :>i u >» y ' * 

1, NumUi uf w utJj »i iit t tn.,.._j =^^r-Acc€ptoble--numWj:r=^ 3t-Too lew— 

D. Comprehen$tfc?ify . ' ^ , 

Easy to understand Difficuli to understand Cannot understand. 

- perseveration of vurds illogical 

perseveration of ideas disorganized 

E. f?eo/ity 

Accurate perception of stimulus or task 

Inaccurate perception of stimulus or task 



F., Style m-A^ 

1. Sentence Sense 

a. /Completeness T^iHes: 

(1) complete sentences — _J 1 

(2) . run-on sentences — 

(3) . sentence fragment;-_^ — 

b. Structure 
(1), simple. 



12) compound 
(3) complex 



(4). compjund/comp!ex_ 

. .c. Types 

(1). ded?.rative ^ 

(2;. Irtterrogative 

(3) . imperative — — 

(4) exclamatory 

2. Tonel/' 

a. intimate b, friendly c. impersonal 

^ord Choice (N = none, F = few, S = some. M = many) 

a. f^ 

formal^r^i^informal colloquial 

b. complexity 
Simple multisyll^glteS^^j^ontraciions. 

c. descriptiveness ^ ^ 
vaguc^^<i!fvK'id figures of speech. 

d. apgrf^pvMrteness ^ 

inexact words, superfluous/repetitions orniciions. 



•Adapted from Iht /nuertfory o/ lVr/»en Exprti^xon and Spelling (Poteet, 1980). 
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